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State Leaders Connecting Learning to Work 


THE ROLE OF DATA AND ACCOUNTABILITY 
IN GROWING YOUTH APPRENTICESHIP PROGRAMS 
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based learning for students to ensure that they gain the skills they need to be successful 
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new youth apprenticeship programs to further expand access to high-quality, career- 


focused pathways for more learners. 


YOUTH APPRENTICESHIPS — when designed with quality and equity in mind — allow students to 
complete high school, access early postsecondary opportunities at no cost, get paid work experience 
alongside a mentor, and start on a path that broadens their options for the future. These opportunities give 
students access to career-focused pathways that provide them with real-world insights into how their 


passions connect with in-demand careers. 


However, the policy and practice of youth apprenticeship are still relatively nascent in the United States, 
necessitating supports and resources for the field. One area that is particularly under-developed is the 
ability of states and local communities to collect and use reliable data to support and improve youth 
apprenticeship programs. The data are lacking for a number of reasons. For one, youth apprenticeships are 
still very new in many states and communities, with the model still being refined, and early data collection 
efforts have focused on gathering student participation and completion data rather than data on outcomes. 
Second, youth apprenticeships often cut across multiple systems and sectors, requiring strong alignment 
across secondary, postsecondary and workforce data systems and a clear sense of which entities are 
ultimately responsible for collecting, sharing and/or reporting data. Finally, there are inconsistencies in 


how these programs are defined and delivered, leading to reporting discrepancies. 


The Partnership to Advance Youth Apprenticeship (PAYA) — of which Advance CTE is a member — has 
developed guiding principles for all high-quality youth apprenticeship programs. One of the five principles 
is accountability — more specifically that student, employer and program outcomes must all be measured 
in a transparent manner. This accountability is imperative for the continuous improvement of youth 


apprenticeship programs. 


PAYA == 


PARTNERSHIP TO ADVANCE YOUTH APPRENTICESHIP 


Created by New America, PAYA works to understand the ways that youth 
apprenticeship programs can be re-thought and developed to meet the needs 
of learners and employers and then build responding solutions. Advance CTE 
is one of the members of PAYA, along with CareerWise Colorado, Charleston 
Regional Youth Apprenticeships at Trident Technical College, Education 
Strategy Group, JFF, The National Alliance for Partnerships in Equity, the 
National Fund for Workforce Solutions and National Governors Association. 
Over the next few years, PAYA will support the work that cities and states are 


doing to grow access to high-quality youth apprenticeship opportunities. 


PAYA is grateful for the support of the Annie E. Casey Foundation, the Ballmer 
Group, Bloomberg Philanthropies, the Joyce Foundation, JPMorgan Chase & Co. 
and the Siemens Foundation. 


The following brief offers information on states’ current capacity to collect and use data around youth 
apprenticeships (and work-based learning more generally) and the steps states need to take to ensure that 
the field can best leverage data-driven decision making to support the expansion of high-quality and truly 


equitable youth apprenticeships. 


In fall 2018, Advance CTE conducted a survey of State Career Technical Education (CTE) Directors to 
understand the quality and effectiveness of career readiness data. The survey was conducted in partnership 
with the Council of Chief State School Officers; Education Strategy Group; the Data Quality Campaign; and 
the Workforce Data Quality Campaign, a project of the National Skills Coalition. It was funded by JPMorgan 
Chase & Co. through the New Skills for Youth initiative. A total of 51 State Directors responded to the survey, 
representing 48 states, two territories and the District of Columbia. This brief draws on the information 
reported through the survey that is specific to youth apprenticeship and work-based learning data. 


For a full overview and analysis of the survey, check out The State of Career Technical Education: 


Improving Data Quality & Effectiveness.” 


How States Define “Youth Apprenticeship” 


Just 20 states report having a statewide definition for the term “youth apprenticeship,’ with significant 
variation across states in how they go about defining these programs. Some states have specific and 
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a broader statewide definition for work-based learning. Other states simply refer to the U.S. Department 
of Labor description of a youth apprenticeship.’ Even in states that have adopted a common definition or 
framework for youth apprenticeship programs, local communities may have flexibility in developing youth 
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defines youth apprenticeships as “earn and learn” work opportunities focused on the manufacturing, 
science, technology, engineering and math industries. Youth apprenticeship program students 
complete at least 450 hours of on-the-job training with a certified employer, while simultaneously 
receiving related educational training through their high school. Participating students typically 
work during the summer after their junior year and during their senior year with a state-approved 
employer. They work with a mentor to learn valuable skills and earn industry credentials and high 
school credit. Youth apprentices also receive training in employability skills, inter-personal/social 
skills, and general knowledge of the world of work.’ All state-approved youth apprenticeships are 
now automatically approved as CTE programs of study. 


, on the other hand, defers to the U.S. Department of Labor definition of youth 
apprenticeship, with a state-level goal for all youth apprenticeships to use as a basis for 
operation. The Oregon Department of Education has published the following: 


The goal of a youth apprenticeship is to get an early start on adult apprenticeship and mirrors 
adult apprenticeship in most respects. Students get first-hand experience in the workplace. 
Local programs provide training based on industry guidelines, endorsed by business and 
education. Students are instructed by qualified teachers and skilled worksite mentors. Students 
are simultaneously enrolled in academic classes to meet high school graduation requirements, 
in a youth apprenticeship related instruction class, and are employed by a participating 
employer under the supervision of a skilled mentor. Students completing a Youth Apprenticeship 


program may be eligible for credits to be applied towards an adult apprenticeship program.” 


Some states have launched youth apprenticeship programs without having established a statewide 
definition for the term. For example, Montana is launching the Montana Youth Apprenticeship Program, with 
support from PAYA, to build out a youth apprenticeship framework. Idaho includes youth apprenticeship in 
its data accountability system through its Every Student Succeeds Act (ESSA) state plan, but the state is 


still developing and formalizing a statewide framework, definition and data collection strategy. 


How States Collect Youth Apprenticeship Data 
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by one or more collection mechanisms. Specifically: 
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a statewide student information system (typically reported by districts or schools); 


Three states reported collecting data through an agreement with the state department of 
labor or other agency; 


Four states reported collecting data through a locally sourced student information system 
by schools or districts; 
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ESSA gives states an opportunity to measure school performance beyond traditional 
academic outcomes. ESSA’s School Quality/Student Success indicator provides states 
the flexibility to implement accountability measures that address both college and career 
readiness. Work-based learning experiences, including youth apprenticeships, are one 
measure that states can use for this indicator. In total, 12 states included a work-based 


learning measure (or equivalent) in their ESSA accountability systems, six of which (Idaho, 


Carolina and Wisconsin) explicitly call out youth 


Kentucky, Maryland, Oklahoma, So 


apprenticeship as a way of meeting a college- and career-ready benchmark.* 


For example, South Carolina is using nine college- and career-ready indicators under ESSA. 
One of the career-ready measures is the completion of a registered youth apprenticeship as 
established through Apprenticeship Carolina. To count under ESSA, the work-based learning 
experience must reach at least 40 hours, be aligned with a CTE pathway and be validated by 
the involved employer. The state ensures quality data collection by requiring that the data be 
validated through a state-defined process. All eligible work-based learning activity must 
include a training agreement that outlines objectives and skill attainment. The agreement 
also must call for an exit evaluation by the employer. South Carolina’s Work-Based Learning 
Implementation Guidelines explain how to report on student enrollment and completion for 


each different kind of work-based learning experience.’ 


Despite the impactful role that youth apprenticeships can have for participants, there is little data to docu- 
ment the programmatic landscape, let alone the actual impact on learners. Quality, verified data on student 
enrollment, retention and outcomes can support the expansion of successful programs and the improve- 
ment of those that are falling behind. The lack of data means that not only are leaders unable to understand 
what youth apprenticeship looks like in their states and communities and who Is — or is not — being served, 
but they also do not have the tools necessary to ensure program quality. Even when states are building their 


apprenticeship data into statewide information systems, the data are still commonly self-reported. 


Furthermore, only 12 states reported having the ability to collect some form of disaggregated youth 


apprenticeship data. 


States and local communities cannot adequately address and close equity gaps in youth apprenticeship 
participation and success without the data to identify such disparities. Collection of these data is often 
what brings attention to gaps and prompts a root cause analyxsis and plan for action. Thus, disaggregated 


participation and outcomes data are vital to program improvement. 


Number of States with Disaggregated Youth Apprenticeship Data 


™@ Number of States 


YOUTH APPRENTICESHIP IN THE STRENGTHENING CAREER AND 
TECHNICAL EDUCATION FOR THE 21ST CENTURY ACT 


In 2018, the Strengthening Career and Technical Education for the 21st Century Act (Perkins V) 
was passed to reauthorize the Carl D. Perkins Career and Technical Education Act. Perkins V 
now defines work-based learning as “sustained interactions with industry or community 
professionals in real workplace settings, to the extent practicable, or simulated environments 
at an education institution that foster in-depth, first-hand engagement with the tasks required 
of a given career field, that are aligned to curriculum and instruction.” All local districts and 
postsecondary institutions receiving Perkins V funding must address how they will support 


work-based learning in their local applications for funding. 


Perkins V incorporates work-based learning as a possible secondary performance indicator. 
States are given a choice for a new secondary program quality indicator; one of the three 
options is “the percentage of CTE concentrators graduating from high school having 
participated in work-based learning.” States that select this option will leverage Perkins V 

to formalize the state’s work-based learning opportunities, as well as send the message that 
work-based learning is a priority for the state. However, to do use this indicator effectively, 
states will need to strengthen their definition of work-based learning and the mechanisms for 
collecting quality data. To learn more about the work-based learning indicator in Perkins V, 
check out Measuring Secondary CTE Program Quality: Work-Based Learning. 


Collecting high-quality and reliable data on youth apprenticeship is just the first step — states also need 
to have mechanisms in place for validating the accuracy of those data, particularly given that the majority 
of states collecting youth apprenticeship data are relying on self-reported data from the local level. 

The survey of state CTE leaders found that more than 20 states have processes in place — such as 

a conducting a statewide audit, reviewing statewide data or leveraging regular program monitoring — 

to validate work-based learning data. A handful of states rely on local validation methods. Among the 

16 states collecting data on youth apprenticeship, only 10 states have formal validation processes in 


place, including two that rely on local validation of data. 


This finding is not surprising given the capacity needed to validate data as complex as youth 
apprenticeship data. States are grappling with how to collect and validate work-based learning data 
more broadly, especially to confirm not only participation and completion but also that the experience 
was meaningful, connected to careers, and led to positive learning outcomes and skills attainment for 
students. Outcomes data can be the most ambiguous and difficult to define but will provide some of 


the most impactful information needed to make data-driven choices. 


STATE EXAMPLE: GEORGIA 


The state of Georgia defines a youth apprenticeship as a “structured combination of school- 
based and work-based learning” that is based on collaboration with industry and results in a 
student receiving a high school diploma, postsecondary degree, or certificate and industry- 
recognized certification for a high-skilled occupation. These programs are required to include 

a partnership of secondary schools, postsecondary institutions, employers, and representatives 
of the greater community; direct connections from secondary to postsecondary; and a combi- 
nation of classroom and on-the-job instruction. Youth apprenticeship is included under the 
larger statewide work-based learning framework, so general work-based learning qualifications, 
such as a minimum of 720 hours of on-the-job training, apply to youth apprenticeship programs. 
However, these apprenticeship programs also have unique components. Youth apprentices are 
evaluated on academic grades, workplace performance, and behavior in the secondary school 


and workplace. 


Before ESSA was passed, Georgia already had an accountability system — the College and 
Career Ready Performance Index — that incorporated work-based learning. Under ESSA, 
Georgia is keeping work-based learning as part of the state’s college and career readiness 
indicator. Specifically, a student must obtain one full credit for a work-based learning course 
that is part of a CTE pathway. Each CTE course is approved by the state and has a standardized 
course code. This setup means that the course can be part of most state-approved CTE 
pathways, is included in the state’s longitudinal data system, and has parameters for 


enrollment and completion. 


Georgia collects work-based learning data through its own platform, C-NET. Each local 
work-based learning coordinator is responsible for the submission of these data, including 
written evaluations of work-based learning participation, rubrics for grading, employer- 
completed evaluations and student portfolios. The state verifies the data through audits 


and monitoring.’ 


Looking Ahead 
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processes may inhibit high-quality programs from being scaled consistently within and across states. 
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decisions is necessary for continued growth and improvement of youth apprenticeships at the state and 
local levels. Quality data provide a clear picture of what is working in a program — and what is not — and 
they offer evidence to prompt improvements. By collecting disaggregated data on program participation 
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on positive enrollment and outcomes are one of the most significant assets for a program. States and 
communities need meaningful data to tell the story of youth apprenticeships and to use as part of their 


student and employer recruitment strategy. 


However, to date, states have begun to collect only entry data on youth apprenticeship enrollment, while 
the ultimate goal should be to expand collection to include outcome data that can be fully disaggregated 
by special populations. Only with these data in hand can state and local leaders ensure that they are 


serving learners equitably with the highest quality youth apprenticeship programs. 


To make the most well-informed data-driven decisions, both quantitative and qualitative data should be 
taken into account. Since most accountability systems lack some of the quantitative data needed for a 
comprehensive analysis of a youth apprenticeship program, qualitative factors can also be used to 


supplement those gaps. 


State and local leaders should take the following steps to strengthen quantitative data collection 


and analysis: 


e Adopt a common, rigorous and measurable statewide definition of youth apprenticeship 
programs; 

e Develop a data definition and business rules that align with the statewide definition of 
youth apprenticeship to support consistent data collection and reporting; 

e Learn what data collection looks like at the local and state levels and where gaps exist; 
and 

e Leverage the opportunities provided by ESSA and Perkins V to implement data 


collection mechanisms. 


To supplement gaps in data collection and support findings, the following qualitative methods should be 
used to gain a comprehensive understanding of youth apprenticeship programs: 


° Identify best practices exercised in youth apprenticeship programs locally, statewide 


ornationally, which can then be scaled and expanded; 


e Gather stakeholder input on program successes, challenges and needs through 


interviews, focus groups and/or surveys; and 


° Conduct periodic follow-up with youth apprenticeship program alumni to learn about 
their employment and education trajectory and how the youth apprenticeship prepared 


them for the future. 


GETTING TO KNOW THE PAYA GRANTEES 


In 2019, PAYA awarded nine grants to support the work of employers, educators, community 
partners and policy leaders in creating or expanding high-quality youth apprenticeship programs. 
The grantees were selected from an applicant pool of 200 from 49 states and Puerto Rico. 
They include: 

e Apprenticeship 502 (Louisville, KY); 

e ApprenticeshipNC (Raleigh, NC); 

e The Birmingham Promise (Birmingham, AL); 

e Career Launch Chicago (Chicago, IL); 

e Early Care and Education Youth Apprenticeship (Oakland, CA); 

e King County Regional Youth Apprenticeship Consortium (Renton, WA); 

e Montana Youth Apprenticeship Partnership (Helena, MT); 

e Texas Youth Apprenticeship Program (Austin, TX); and 

e PPL Learn and Earn to Achieve Potential (LEAP) Initiative (Minneapolis, MN). 


These sites will be supported by the PAYA grant over a 17-month period that started in May 2019. 


https://www.newamerica.org/education-policy/edcentral/youth-apprenticeship-definition-and-guiding-principles/ 


https://careertech.org/resource/state-cte-improving-data-quality-effectiveness 


https://www.apprenticeship.gov/faqs 


https://www.dllrstate.md.us/employment/appr/youthapprfags.shtml 


https://www.oregon.gov/ode/learning-options/CTE/resources/Documents/Understanding%20and%20Accessing% 
200regon%20Apprenticeships.pdf 


https://careertech.org/resource/making-career-readiness-count-2019 


https://cte.careertech.org/sites/default/files/Learning_Leadership_Profile_2018.pdf 


https://www.gadoe.org/Curriculum-Instruction-and-Assessment/CTAE/Pages/Youth-Apprenticeship-Program.aspx 


* Ibid. 


